API Model for Messenger Service
Learn about the main data entities and API endpoints needed to build a chat service like Messenger.
In the previous lesson, we decided on the architectural aspects of designing the Messenger API’s communication protocols and data formats. In this lesson, we will describe the endpoints, data entities, and the message format required for a chat between clients.
Base URL and API endpoints#
We will use the following URL for designing the Messenger API. The api.messenger.com is the host URL followed by the version v1.0 and the service.
The following illustration represents the operations and their associated endpoints:
Let's go over the purpose of the endpoints below:
WebSocket connection: This endpoint is used to establish a WebSocket connection with a chat server.
Retrieve or upload an asset: This endpoint is accessed to retrieve or upload a media file.
Retrieve a paginated list of messages: These endpoints are targeted to retrieve a paginated list of messages.
Remember: We don't have an endpoint for a live and offline chat because we use WebSockets for such communication. We will shortly see how that interaction between users takes place.
The message format for API endpoints#
In this section, we’ll discuss data entities, requests, and response structures for each functional requirement. We will also look at how a WebSocket connection is established to send one-to-one messages. Let's start our discussion with data entities.
Message data entities#
Following are the key data entities that are exchanged between the client and server:
Establishing WebSocket connection#
Before sending a message, it is mandatory to have a WebSocket connection established with the chat server. Therefore, we must establish a connection with the chat server represented by the v1.0/chat endpoint. As shown below, the encoding parameter encoding=text shows that the messages will be exchanged in the UTF-8 encoding after the successful connection.
HTTP method: The WebSocket connection with the chat server is established by simply sending an HTTP
GETrequest with some essential headers.Request format: In the request body, we set the value of the
Connectionheader toUpgradeand the value of theUpgradeheader towebsocket, indicating that we want to upgrade our current HTTP connection to the WebSocket protocol. The request is shown in the following code widget:
The Sec-WebSocket-Version specifies a WebSocket version the client aims to use when communicating with the server. In the initial connection upgradation process, the client mentions the WebSocket versions that it can support via this header. For example, the client request could contain version 13 as shown below:
Sec-WebSocket-Version: 13
If the server supports version 13, the response from the server does not include the Sec-WebSocket-Version header. However, if the server doesn’t support the requested version, the response contains a 426 Upgrade Required error, and the Sec-WebSocket-Version header shows a list of values that a server supports. For example:
Sec-WebSocket-Version: 7, 8
The client then repeats a request to the server with the latest supported version amongst the list shared by the server, that is, version 8.
After the successful WebSocket connection establishment, the communication between the client and server starts using WebSocket version 8.
Response format: When the connection is successfully established, the server returns a response, as shown below. In the response, we see the status code
101, indicating that the WebSocket protocol has been successfully upgraded.
The following figure illustrates the process of WebSocket connection establishment:
In the above figure, we establish an HTTP connection with the chat server, which is then upgraded to a WebSocket connection.
Note: As soon as a WebSocket connection is established between a client and chat server, the latter communicates about the connection to the WebSocket manager, which updates its mapping table.
Live communication#
After establishing a WebSocket connection, let's see how text messages are sent over this connection in the JSON format. Before sending the data over the connection, the WebSocket API converts it to the UTF-8 encoded text format. Let's start with a message without an attachment.
Message without an attachment#
When the clients establish the WebSocket connection with the server, they can send and receive text messages to each other. A sample text message, along with its metadata, is shown below:
When the server receives the message, it will send an acknowledgment back to the sender showing that it has received it. Line 9 in the following code widget shows that the attribute receivedServer is set to true in the response from the server. Similarly, the server responds with the delivered and read attributes as true when the receiver receives and reads the message.
Points to Ponder
Question 2
What is the format of a message if we send multiple attachments in a chat?
If a message has multiple attachments, we can represent them via a JSON list—for example, attachmentURI: ["URI-01", "URI-02", ...] as shown below:
{
"connection_id": "rLCkwH/SKGAsO9H/ZShrBAFTDKU=",
"payload": [
{
"senderId" : "Rtnukkald=",
"recieverId" : "IJUkalxUl=",
"messageId" : "Pqwnkilsx=",
"timestamp" : "9:47PM 12-1-2023",
"attachement" : true,
"attachementURI" : ["/v1.0/asset/{id-01}", "/v1.0/asset/{id-02}", "..." ],
"messageText" : "Hello, Welcome!"
}
]
}
2 of 2
Retrieving offline messages#
When the presence server determines that some user's status is offline, the messages intended for them are stored in the database and marked undelivered. When a user comes online, the presence server receives a heartbeat event from them, the chat server is notified, and the WebSocket connection is established. In the next step, the messages are delivered via the WebSocket connection in the format shown below:
Note: he WebSocket protocol uses some control signals, such as ping and pong, that work as the heartbeat to check that the client is still responsive.
Point to Ponder
Question
How does searching a list of messages work?
There are two scenarios to search a list of messages for a query:
-
Client-side search: In this approach, the user’s messages are searched for the query locally on the client side. While the most recent messages are cached, older messages may be stored and later retrieved from a local database.
-
Server-side search: In this approach, the search API is utilized using
GET, where the user’s query is sent to the backend servers. At the backend, the user’s messages are searched for the query, and a relevant paginated list of messages is retrieved.
Note: The server-side search approach is generally used in tandem with the client-side storage. This is because the client shouldn’t be burdened by storing all the previous messages. However, clients often revisit recent messages, and therefore, for a good user experience, it makes sense to keep recent messages in local storage.
Retrieving a paginated list of messages on alternative devices#
While messages may be retrieved locally from a database, they are not readily available on alternative devices. In that case, the client may GET messages in the form of a paginated list. Let's discuss the request and response size for retrieving a paginated list of messages on other devices.
Note: We have a detailed lesson on the different types of pagination. You can revisit it here.
HTTP method: To retrieve a paginated list of messages, we use the
GETmethod.Request format: In the query string, we mentioned the
userIDrepresenting the sender's window, from whom the messages are retrieved. The query string contains theafter_idandlimit, which shows the message ID and the number of messages we want to receive, respectively.
Response format: The response from the server contains a list of messages in the body, as shown below:
To retrieve a paginated list of messages, the body in each response consists of messages along with the message metadata, such as senderId, receiverId, messageId, timestamp, and so on.
Remember: We will use this endpoint only to retrieve a paginated list of messages. To perform real-time chatting via the alternative device, a WebSocket connection will still be required to establish two-way communication with the server.
Failed requests#
The following table indicates different status codes in case of errors while performing various operations relevant to sending or receiving messages:
Error Responses
Status Code | Phrase | Description |
| Bad Request |
|
| Unauthorized |
|
| Forbidden |
|
| Not Found |
|
| Method Not Allowed |
|
| Payload Too Large |
|
| Upgrade Required |
|
| Internal Server Error |
|
Summary#
A summary of the request detail for each endpoint is shown in the table below. The two-way chat is performed via WebSocket protocol, and the rest of the operations are performed via HTTP methods.
Operations | Endpoints | Data Entities | Response (Status code, body) |
WebSocket connection |
| No data entities |
|
Retrieve an asset |
|
|
|
Upload an asset |
| The media file, and the relevant metadata is passed in message body |
|
Retrieve a paginated list of messages |
|
|
|
Messenger API Design Decisions
Messenger API Design Evaluation and Latency Budget